Skip to content

chore(ci): general speed improvements - #215

Open
kyteinsky wants to merge 2 commits into
mainfrom
chore/reduce-context-size
Open

kyteinsky wants to merge 2 commits into
mainfrom
chore/reduce-context-size

Conversation

@kyteinsky

Copy link
Copy Markdown
Contributor

also a tool to check the token usage of system prompt + all tools

🤖 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@kyteinsky
kyteinsky requested review from janepie and marcelklehr July 21, 2026 14:59
@kyteinsky
kyteinsky marked this pull request as draft July 22, 2026 11:41
Signed-off-by: kyteinsky <kyteinsky@gmail.com>
Assisted-by: Github Copilot: claude-opus-4-8
Signed-off-by: kyteinsky <kyteinsky@gmail.com>
@marcelklehr
marcelklehr force-pushed the chore/reduce-context-size branch from 262a689 to 9513f1f Compare September 16, 2026 09:05
@marcelklehr
marcelklehr marked this pull request as ready for review September 16, 2026 09:06
@kyteinsky

Copy link
Copy Markdown
Contributor Author

had converted to draft since gemma 4 E4B didn't seem to work quite as well with llm2, there were tool calling issues, maybe that was fixed?
or we should try some other small llm here.

@marcelklehr

Copy link
Copy Markdown
Member

As you like :) I thought it would be a nice change to have faster CI, but if it doesn't work....

@kyteinsky

Copy link
Copy Markdown
Contributor Author

for sure
we could try something like this: https://huggingface.co/XHToken/Spark-X2.5-1.7B or the 4B version, seems quite good for the size
it does have llama.cpp support but would need some testing with llm2 still

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants